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ABSTRACT 


Determining the proper emphasis of curriculum contents 
as well as judging the value or worth of training programs 
has become an important problem. The purpose of this paper 
is to demonstrate the usefulness of task analysis in 
measuring the effectiveness of training courses based on 
mmienexvent CO Which curriculum contents are job oriented. 
In this regard, parametric and nonparametric statistical 
procedures are discussed as well as a matrix method of 
evaluation. A general methodology to include the opera- 
tional significance of the data is also included. 

Phig@ivuaireiive tne results Of a small scale experiment 
are presented. This experiment was conducted to determine 
the most valid questionnaire associated data collection 


method. 
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IT. INTRODUCTION 


The increasing technology of military developments has 
Paeourvassing the Military services' ability to supply 
competently trained manpower for meeting the Knowledge and 
Zaomeecal skill requirements for particular military jobs. 
While the adequacy of military training is dependent on the 
existence of effective procedures for developing and main- 
taining the appropriate instructional content, a means of 
judging the value or the worth of the training program has 
PaemPpecome ampertant. It is, however, quite misleading to 
Peeeteromony Eraining model or training curriculum with a 
weeren eltfective-inetfective judgement, since training 
programs may be considered adequate under certain circum- 
stances and inadequate under others. 

i“e@atiwaeeempee sO achieve a high positive correlation 
between job proficiency levels and the associated training 
fonusgeall of the military services are striving to estab- 
PoMeiiourieladonal SyYStems around a solid foundation of job 
SevemimimemepOrterays Valid instructional objectives. Since 
job data refers to a statement of the work that an individual 
Pie evornOormsnwom the job, the ideal source for the 
collection of this data is the incumbent performing in his 
assigned billet. The technique of identifying and collect- 
ing these essential elements of the job is called task 


ee ar oemmocsucally . by utilizing a carefully structured 





questionnaire containing a composite list of the probable 
tasks included in a job, as well as a rating scale for 
classifying tasks as to their degree of necessity, the 
extent to which certain elements of the job constitute an 
individual's performance can be precisely identified. Once 
a precise identification of the emphasis devoted to specific 
duties on the job is achieved, training objectives can be 
adjusted in order to bring training emphasis more on target. 
Task analysis, therefore, is useful in determining what 
training should be given and when it should be given in 
order to be timely. 

Prior to the introduction of task analysis procedures, 
forse CONtLeCNLS were primarily constructed to include those 
areas Of instruction which were thought to be required for 
adequate on the job performance. Additionally, no quanti- 
Tative methods were available for evaluating training 
programs within the military services. With the advent of 
task analysis and the subsequent numerical taxonomy of job 
elements, sufficient data has become available for the 
Wiavezauron Cl Sbavistical techniques in analyzing the 
Sitecruaveness Of military training. 

tipesmpaper 1S pased On the continued interest of the 
United States Marine Corps in the area of task analysis and 
job data utilization, as well as the recently initiated 
DrOcrenmoen mnliliGary occupational studies. In particular, 
the primary objective of the paper is to demonstrate the 


usefulness of task analysis in the evaluation of military 


ee 





training programs. More specifically, a task analysis 
performed by the investigator is employed as a data base 

for illustrating some of the appropriate statistical tech- 
niques available for evaluating the effectiveness of three 
Powe a Marine Corps training courses. Additionally, an 
approach toward determining the operational as well as the 
Statistical significance of the effectiveness of these three 
femmeses 1S presented. 

As a secondary objective, validation techniques and 
statistical comparisons are offered for three task analysis 
Gata collection methods applicable to the job inventory 
Mecsvlonnaire, In this regard, the results of a small scale 
experiment designed to determine the most valid questionnaire 
associated data collection method are presented. 

It is hoped that this research will allow quantitative 
Peevoereas qualitative training curriculum analysis to become 
amore standard and definitive process within the Marine 


Corps. 


A. SCOPE AND LIMITATIONS 

SHemuornalearine, COrps training program conducted at 
PiemviazinenwCOrps oer vice support Schools, Marine Corps Base, 
Camp Lejeune, North Carolina, is used to demonstrate the 
Meer iiness of task analysis in the evaluation of military 
Teint cm erciiti ne program is subdivided into three 


separate courses designed to allow trainees to advance from 


one course to the next as they advance in rank. Basically, 
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Piemprolcram 1S structured fo include a basic, an interme- 
diate, and an advanced level course, and is devoted to 
training Marines from the entry level rank through E-7 in 
eeemarea Of SUPpply administration. A description of the 
subject curricula as well as the rank requirements for each 
memmoresenved in Appendix A. 

Miemeavassaseuvespe used for illustrating the analysis 
techniques applicable to training evaluation was collected 
from a representative sample of Marines serving with MOS 
(3041) supply administrative man manual and MOS (3042) supply 
aogministrative man mechanized. Since the basic training 
course is designed to prepare Marines for filling manual 
supply billets, only Marines with MOS (3041) were used to 
provide data for the evaluation of this particular course. 
However, the intermediate and advanced level courses are 
devoted to preparing Marines for both manual and mechanized 
supply positions; therefore, Marines with MOS (3041) and 
MOS (3042) were employed to provide data for the evaluation 
of these two courses. The data has been aligned into three 
distinct categories representative of the Marine's most 
recent course completion. This arrangement provides data in 
an easily accessible manner for evaluating the individual 
courses as well as the overall training program. 

HicemMunianeevartenles cOonuributing Co successful job per- 
formance can be categorized into two specific areas of 
human factors oriented elements. One of these categoriza- 


MoOonmoMmclhuicdes phe actual units of work that form a 
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Significant part of a job and are referred to as performance 
tasks or task elements. The other categorization includes 
Mme worker characteristics required for satisfactory job 
performance. These characteristics are not related to 
Peoeriac Gducvies but include such factors as physical strength 
and mental requirements. A job inventory questionnaire was 
selected as the device for collecting job data and was 
designed to include these two categorizations of human 
Factors oriented elements. 

The performance task portion of the questionnaire con- 
sists of actual task elements which represent the essential 
part of the research. Data obtained from responses to these 
Selected job description statements is used to demonstrate 
techniques for determining whether the skills needed for 
satisfactory performance in an administrative supply billet 
Peemeematabene with the course contents of the associated 
formal training Cit Memmlumtwen ome oertormance task portion 
of the questionnaire developed by the investigator is 
illustrated in Appendix B. 

Mitmweugh the primary emphasis of this paper is devoted 
to the analysis of actual performance task data, the worker 
characteristic segment of the questionnaire is representative 
Sumecamarcempe to quantify the human characteristics required 
BoRoOMerime thie SUbply field. This portion of the question- 
naire was developed by Professor Gary K. Poock, a member of 
the Operations Analysis faculty at the Naval Postgraduate 


Seheol and is displayed in Appendix C. Obviously, the data 
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obtained from the worker characteristic section of the 
SeesuloOnnaire does not lend itself to training curriculum 
evaluation but was useful in providing an additional data 
base for the validation and comparison of different data 
Pemecti10n methods. The information obtained from this 
particular portion of the questionnaire would be most 
appropriately applied to an analysis of MOS assignment 


procedures. 


Eee SoSUMPTIONS 

iiew basis fOr the quantitative evaluation of training 
effectiveness presented in this paper rests on the assump- 
tion that the most important tasks performed by an individ- 
ual on the job are the ones which acowey Ge Nay Or pore von 
of his time. Accordingly, these elements of the job which 
occupy the most time are the ones in which students should 
receive the most intensive training. Basically, a student 
should be trained to perform those tasks which will be 
meauiredwect fam in the field. Of course, the more specific 
the purpose of the training program in terms of performance 
eC erememats for paruicular military jobs, the more appro- 
priately can the effectiveness of the program be determined 
by measuring the emphasis devoted to specific tasks by its 
Precuaveswee Ine training Objectives of many military sehools 
are established to prepare graduates for specific job 
functions, while others are purposely constructed so that 
the dependence of job performance upon the training program 


1s quate indirect. 
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More specifically, the approach to be taken in this 
paper implies a direct relationship between the training 
curriculum and the oe This approach represents the first 
disadvantage in the assumptions presented above in that 
meroearrect relationship is not characteristic of all 
Peerning programs. Curriculums devoted to providing general 
background type knowledge, or those normally associated with 
higher level and more complex positions, often intentionally 
avoid a close relationship with specific elements of the 
OL . 

Another disadvantage associated with the assumptions 
presented above is related to the degree of difficulty 
Mmigieienc In E€ach of the elements of the job. Skills which 
Peemmooncasitly learned Should, in most cases, receive more 
emphasis in the course content. In this regard, some 
ememiemus Of @ job, although performed less frequently than 
others, require more intensive instruction to produce the 
desired proficiency level of the trainee. To eliminate 
moiecetesuriction., relationships between job proficiency 
levels and training emphasis could be established for the 
essential elements of the training program. 

homameeranole or such a relationship consider the 
mMipemnecenecal curves Shown in Figure 1. These curves repre- 
sent the job proficiency level versus the training emphasis 
devoted to two maintenance oriented tasks. Since rotating 
truck tires is relatively simple and learned rapidly, 


maximum proficiency is achieved quite quickly. In contrast, 


ie 
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conducting vehicle safety inspections 1S a complex task 
which is learned more slowly. Accordingly, improvement in 
m@ortciency Can be obtained from additional training time. 
As an example of this relationship, consider a training 
program which devotes five training units to rotating 
truck tires and five training units to conducting vehicle 
safety inspections. Suppose the program was developed on 
the basis of a survey demonstrating that approximately eaual 
times are devoted to these tasks on the job. Inspection of 
the graph indicates that if two training units were elimin- 
ated from rotating tires, no reduction in proficiency would 
eccur. These same two units of training could then be 
meplted to increase the proficiency level of the more com- 
plex task by a significant amount equal to (d). Altering 
Pae, Curriculum in this way certainly represents an improve- 
ment in the overall effectiveness of the program. 

eae teae a the aspect of task suitability for "on the 
job training" emphasis is representative of still another 
disadvantage with the assumptions employed in this paper. 
As an example of this aspect of training to the attainment 
of job proficiency levels, consider the hypothetical curves 
shown in Figure 2. The graph suggests that two independent 
ess possibly occupying equivalent amounts of actual ver- 
formance time are not equal in their applicability to 
training on the job. Task number 1 is relatively simple and 
iidteabes the achievement of a high proficiency level in a 


very short time period. In particular, this task is 


et 
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Seon vely Suitable for on the job training and a likely 
Pageaave Lor E€limination from the formal curriculum. Task 
momecer 2 is more complex and does not appear to be appli- 
eaele LO training techniques of this type. 

Although the concepts just examined have substantial 
theoretical value and appear to be feasible on the surface, 
Baey represent several very difficult obstacles. The 
meelusion of these notions would require a variety of arbi- 
iMmeetey aecisions beyond the realm of this paper. It is, 
however, important to understand that deficiencies exist in 
the subsequent evaluation procedures if the previously 
mentioned complex issues are not included. Additionally, 
it is important to realize that accepting the basic assump- 
e2ons will produce valid training program evaluations 
provided that qualitative considerations are applied subse- 


event to the quantitative analysis. 


eae) METHODOLOGY 

The evaluation techniques to be demonstrated are based 
pomesuavistical comparisons of the following two sets of 
data: 

1. The relative importance of the tasks involved in 
the job, 

2. The training emphasis devoted to the tasks involved 
nm Che Job. 

The first set of data was obtained by utilizing the task 
Baventvory questionnaire in conjunction with a personal inter- 


view and actual observation of the incumbents' performance 


He 





mame JOO. Basically, individuals were required to report 
maenwextenue tO Which the tasks listed on the questionnaire 
Seeup1ed their time. In keeping with the basic assumption 
of this paper, those tasks which occupied the most time 
femenconstacred to be the most important. To limit the 
selection of responses available for reporting the extent 
memeirteh Specific tasks occupied their time, a four point 
femection scale was used. Incumbents considered each task 
with the investigator and were required to explain their 
selected response from the four point scale. It was 
anticipated that the required explanation would insure a 
complete understanding of each question, while at the same 
time limit the workers from over responding. The question- 
Maire employed and the associated scoring procedure is 
illustrated in Appendix C. Scores were converted to per- 
eentages to provide more meaningful data in nature of the 
feporisons we be made. A computer program was developed to 
eemsolidate and display the data in an orderly manner. The 
task analysis questionnaire was administered to 95 enlisted 
Marines ranking from E-2 through E-8. Once collected, the 
Gata was classified in accordance with the Marines' most 
So -VemcoulccecomMpletion. AS previously mentioned this 
Slassification provided data in an easily accessible manner 
for evaluating the three courses as well as the overall 
Peocrame Tne results of this classification proviced 29 


Subjects for the basic course evaluation, 50 subjects for 
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the intermediate level course analysis, and 16 subjects for 
Pmenwevaluation of the advanced course. 

The second set of data was obtained from theoretical 
Mmmeticiency levels specified by the directors of the train- 
ing program. 

Comparisons of these two sets of data will determine if 
the training program is devoted to placing emphasis on the 
appropriate instructional areas. Statistical techniques are 
used to illustrate some of the available procedures for 
6€valuating the three courses on an individual basis. The 
eaeweevcwObecnese individual course evaluations are then 
serenaded in order to demonstrate methods for determining the 


ewuecuiveness of the overall program. 


D. TASK ANALYSIS DATA COLLECTION METHODS EMPLOYED 

Although a variety of task analysis data collection 
Paaiecome wise voce employed in this study are representa- 
Give of the procedures applicable to the job inventory 
questionnaire. The questionnaire permits extensive sampling 
of military occupational specialties in a relatively short 
Doceoreuaime., © Addigionally, the questionnaire oriented 
procedures are economically administered and allow large 
Maosea Ol dava to be reduced for computer storage, display 
and analysis. For these reasons it is estimated that the 
data collection methods employed, although far from exhaus- 
tive, are representative of those feasible for Marine Corps 
efforts in this area. The following are the task analysis 


ieomiivenvory procedures utilized in this paper: 
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Me cotoonal inverview and actual observation of an 
magaviaqual's on the job performance. 

2.) Distribution of the task analysis inventory 
Pest Lonnaire in a manner not permitting observation or 
interview of the job incumbent. This procedure requires 
fmarvidual completion of the questionnaire with no explan- 
freon Of the task statements. 

CevlcwmlnZaprtom ol the task analysis questionnaire by 
meemcers and Senior stati non-commissioned officers serving 
mea SUpervisory capacity. For this method job data was 
fimeoined by Supervisors reporting the job characteristics 
mequired of their subordinates. 

tiwecnesdavea, collected using the questionnaire in 
conjunction with a personal interview and observation of the 
job incumbent was used in the evaluation procedures demon- 
meraved in this paper. The other data gathering procedures 
were useful in determining the most valid method for acquir- 


moe Job inventory data. 


ee 





toner CAL LESPTING IN GENERAL 


Memeo cscmol sae Graining course G€valuation is to 
provide information regarding the effectiveness of the cur- 
riculum content. Of particular interest here is to examine 
meavetmiavive approaches toward determining whether or not 
Training is being conducted in the appropriate instructional 
ereas and if students are of the proper rank in order for 
Che training to be timely. As an example of the concept of 
momely training, consider a motor transport mechanic who 
av some Time in his career may be required to employ manage- 
menc techniques in the operation of a large repair facility. 
Since these management techniques will probably only be 
meqguired of senior enlisted men, a timely training program 
Perrot provide instruction in this area for the entry 
mevel or even the intermediate level ranks. What seems to 
be needed here is a good method for determining the effec- 
tiveness of course contents while at the same time providing 
@ foundation capable of incorporating the ideas mentioned 
Peovel into an Evaluation of the overall program. Possible 
approaches toward achieving this method include the use of 
Menparametric as well as parametric statistical techniques. 

Molde wememstavicotetcal test 1s a test in which 
specific assumptions about the parameters of the sampled 
population are made, such as UW, = Uo OF 0, = Oo; whereas 


a nonparametric test makes no assumptions about the value 
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of the parameters in the sampled population. It is impor- 
mame GO Gistinguishn the Giiference in the assumptions which 
must be made when testing parametrically vice nonpara- 
metrically. The basic assumptions of eis parametric test 
include independence of observations, underlying normal 
distribution of the sampled populations, and homoscedacity 
of the population variances. The assumptions associated 
with nonparametric tests include only the independence of 
observations, and that the sampled populations be continuous. 
When judged by the criterion of power efficiency, nonpara- 
metric tests are often superior to their parametric counter 
parts when all of the assumptions required of the parametric 
tests can not be met [1]. 

In keeping with the primary objective of this paper, 
which is demonstrating the usefulness of task analysis in 
miemevaluation of military training, an attempt is not made 
iO determine whiten test or even which type of test is the 
Pi erwOoresiicnean analysis. Rather than make such a deter- 
Mination, a variety of applicable evaluation procedures are 
Presenved, in addition to presenting applicable statistical 
evaluation procedures, the concept of operational effective- 
ness and a general methodology for standardizing military 


training evaluations are examined. 
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iit NONPARAMETRIC EVALUATIONS OF TRAINING EFFECTIVENESS 


Pee «Oo LESTS 

The applicable tests selected to demonstrate nonpara- 
metric statistical procedures for evaluating the effective- 
ness of the Marine Corps training courses are the Kolmogorov- 
Smirnov Test and the Spearman Rank Correlation Coefficient. 
A matrix method of evaluation, also a form of nonparametric 


testing, will be examined in a separate chapter. 


B. KOLMOGOROV-SMIRNOV TEST 

Perhaps the most huerestic of the statistical tests is 
mee KOlmMOforov—-smirnov One Sample Test, often referred to as 
tne Smirnov Maximum Deviation Test. The test statistic is 
the maximum deviation between two empirical distribution 
functions [4]. Since training effectiveness in this paper 
eee determined by evaluating the extent to which course 
contents are job oriented, the actual proportions of job 
time devoted to the different task elements are used as 
mee standard, Therefore, the distribution of job time over 
the task elements is considered the theoretical fixed cumu- 
MaGgive GiSsebribution. Basically the data in each category, 
iye., basic, intermediate, and advanced, was collected using 
the four point rating scale displayed in Appendix C and was 
then individually averaged over every task statement on the 
questionnaire. These averages were then converted to per- 


centages in order to provide the average proportion of time 
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devoted to each task element by the most recent graduates 
of each of the three Marine Corps training courses. More 
specifically, a fixed theoretical cumulative distribution 
of job time was derived for each of the three categories of 
actual job data. 

The other empirical distribution was determined from the 
directors of the training program and represents the propor- 
tion of time devoted to the task elements within each of the 
moeee courses. So at this stage of the procedure, a theo- 
meraeal fixed cumulative distribution of actual job time and 
a cumulative distribution of training emphasis were available 
for each of the training courses. 

Memeconpube the test statistic, list the values of the 


cumulative distribution functions of training emphasis and 


job time in order of the various task elements. The test 
mmaeistic, D, is max|d,|. Where: 
n n 
Ga dae 2 TT. , i eee este. DC 5 
Bessie) 1-1 + 
and J = average proportion of actual job time devoted 
to task element i. 
Ts = proportion of training time devoted to task 


element i. 
Testing is conducted under the null hypothesis, Hots eat 
the two distributions are the same and the alternative 


Mmypothesis, H PMc mtWwe Gictribpuraions are not the same. 


ge 
It is important to note that the Kolmogorov-Smirnov Test 


used in the manner explained above does not meet all of the 
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prerequisites required of nonparametric statistical tests. 
The data is cast into intervals defined by each of the 
tasks listed on the questionnaire, and therefore, cannot 
meet the continuity assumption required for this nonpara- 
metric test. Additionally, the results of the Kolmogorov- 
Smirnov Test used in this way voroduce a somewhat pessimistic 
appraisal of the effectiveness of the training courses, 
since the value of D will be at least as great as the 
maximum difference between the job time versus training 
emphasis on any one task element. The test, however, has 
ieeliosve appeal in that it considers proportional differ- 
ei@comom Gistinet task elements as well as runs of task 
Scavements which display discord between the respective 
PeOpOorvions. 

eoieeeiiicwration OF the manner in which this test 
Was employed, consider the data associated with the Basic 
mupply Course. An explanation of the nature of this course 
moepresenved in Appendix A. The data points of the two 
mmiueauive AlStributions as well as the d,'s are shown in 
Table I. Similar tables for the remaining courses are 
presented in Appendix D. Table II illustrates a general 
plot of the two cumulative step functions associated with 


the Basic Supply Course. 
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TABLE I 


CUMULATIVE DISTRIBUTION DATA POINTS IN KOLMOGOROV- 
Serve irom WitrH BASIC COURSE DATA 


mask No. Training (CDF) JoOpmUCDI ) dq, 
a 02 0 Odes BO AG 2 
o eOe .0036 POO 
3 woe sO256 O50 
02 20250 O50 
> mee ales BO dees 
6 Ke Ore .0086 
7 28 plod .0984 
8 Ee S26 sa 
9 oon ROSS) 22145 
LO ~56 » 3346 ~2254% 
le 56 . 3666 ~1934 
ee ao 7S ets 
es Ee ~ 3986 lee ee) 
14 ~56 a Call - 1009 
aS Ao S06 .0664 
16 -60 .5890 SOsraeG 
nny oO .6566 .0566 
18 ~60 .6993 .0998 
19 Ao . 7456 .1356 
20 no ~7634 .0934 
21 68 ~7794 .0994 
22 ls ~8257 10257 
ae ~92 8542 .0658 
pak ~96 Pook .0809 
25 melo veveub .0809 
26 ~ 96 ~ 8846 Wyse 
Zi ~99 moe Te .0627 
2G eo 9842 Oa Sis 
29 1.2.00 if, CO00 0 
30 I 0'O Peo 0 
ol ie O10 L70000 0 
B2 OC Oooo 0 
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TABLE II 


CUMULATIVE DISTRIBUTION STEP FUNCTIONS IN KOLMOGOROV- 
Oi tewiln BASTC COURSE DATA 


1 : 
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is Training 
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22 
D ag) I 20 25 Sie 


Task Elements 


D occurs at i = 9 where 


dy = .56 - .3346 = .2254, 


Mme compared with the critical value of .1613 it is 
feeotole to reject the null hypothesis at a significance 
Mevel of .01. The values of D for the courses under 
examination are shown in Table III. The null hypothesis 
when applied to the remaining courses may also be rejected 
Bee une .O1 significance level. 

A rejection of the null hypothesis at the .01 signifi- 


Cance level for all three courses indicates that training 
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SUMMARY OF THE KOLMOGOROV-SMIRNOV TEST RESULTS 


D 
Basic Course ee 2 el 
Administrative Course oe 4 
Senior Course me 1 36 


emphasis is not devoted to the appropriate instructional 
areas. However, more important than the statistical sig- 
Naricance determined by the Kolmogorov-Smirnov Test is the 
manner in which the test presents the data. The examination 
of the two cumulative distributions from a more operational 
standpoint, may indicate feasible adjustments for bringing 
training emphasis more on target. 

As an example of some of the comparisons that can be 
made, consider the illustration of the two cumulative dis- 
tribution functions presented in Table II. Tasks nine 
through fifteen represent a trend toward overtraining. The over 
emphasis devoted to training on these tasks may be more 
appropriately applied to tasks twenty through twenty five, 
where an undertraining trend is indicated. Similar compari- 
sons can be made among the three courses in order to locate 
possible adjustments for improving the training program on 
an overall basis. More elaborate comparisons could be made 
for evaluations containing a much greater number of perform- 


ance tasks, which could be classified and sequenced into 
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distinct instructional areas. Nevertheless, the Kolmogorov- 
omirnov Test, as illustrated, is useful in providing an 


indication of the effectiveness of the training courses. 


C. SPEARMAN RANK CORRELATION COEFFICIENT 

Of all the statistical tests based on ranks, the Spearman 
Rank Correlation Coefficient is perhaps the oldest and the 
most widely known. The test statistic ris a measure of the 
degree of association that exists between two groups of 
data [5]. 

To compute r it is necessary to establish two separate 
categories of data based on the assignment of ranks to the 
tasks listed on the questionnaire. The first ranking is 
based on the relative importance of the tasks to satisfac- 
tory performance on the job while the second is achieved 
from the emphasis devoted to the tasks in the training pro- 
gram. The task determined the most important to the job is 
assigned a rank of 1 for the job data category while the 
task receiving the most intensive training emphasis is 
assigned a rank of 1 for the training data category. The 


remaining tasks are ranked accordingly. 


The test statistic 


N 
ace 
i=1 > 
a 3 
N~- -N 
where: 
dad. = the difference between the two ranks assigned To 


task element i. 


Salt 





to 


The value of r ranges between O and l. 


the number of task elements. 


The highest 


moeo ble degree of association is 1, and can be achieved 


meeeand only 


if, each task element receives the identical 


ranking within the two categories of data. 


menene Preporvion of ties within the rankings of either 


category is large, 


value of r 


where: 


or 
il 


S 
ll 


The variable 


variable Y represents the job data. 


Pye ameemrecc lon factor, 


it becomes appropriate to adjust the 


MieCmeGmicoerienmr actor, 


n tae - t. 
oe ye 


, X,Y 
1 12 


ak 


the number of observations tied at rank i. 


the total number of ranks at which tied obser- 
vations exist within a given category of data. 


X represents the training data, while the 


Witthe ene s correction 


factor incorporated one can use the formula: 


where: 


The rankings 


ricula under 








N 
x°¢ yo - nc 
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Coated mnGtache Computation of r for the cur- 


observation are listed in Appendix E. 
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Additionally, it is possible to test the significance 
of the value of r or r. This can be done by testing the 
mobi hypothesis, Ho: that the two variables under observa- 
imeem are NOU associated against the alternative hypothesis, 
Hy: that there is a measure of association between the two 
variables. The number of task elements being greater than 
mmeaevermines the use of the significance test for large 
samples [5]. The significance of an obtained r under the 


nue hypothesis may be tested by: 





Memeuesc Statistic, t, has the student's t distribution with 
degrees of freedom = N-2, for large values of N. 
Table IV illustrates a summary of the results obtained 


by using the Spearman Rank Correlation Coefficient. 


TABLE iV 


SUMMARY OF SPEARMAN RANK CORRELATION COEFFICIENT COMPUTATIONAL 
Deo ool sioreliME VERSUS TRAINING EMPHASIS 





r r t 
Basie BOcuUrse nye) noon 5.446 
pMomimtapragive Course aioe ao eo O 
Senior Course . 366 oy 2D 


Ssinee the values of r are so closely related to r, the sig- 


nificance test was only conducted on the uncorrected measures 
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femeoooctaui1om. Ihe results indicate that the null hypo- 
thesis can be rejected in the following instances: 

1. Basic motes are Peowogimeamtece Level of ~001 ; 

PeAcministrative Course at a significance level of .20; 

3. senior Course at a significance level of .05. 
me rejection of the null hypothesis indicates that a measure 
of association does exist between training emphasis and job 
importance for the tasks listed in the questionnaire. 
mmererore, the values of r computed for the three training 
feumses are Statistically significant at the levels shown 
BOVE . 

In addition, qualitative analysis is required here in 
order to determine if the values of r are close enough to 
merreorder for the training program to receive an effective 
rating. The establishment of a standard more easily 
obtainable than a correlation coefficient of 1 may prove 
Peper iate in order for this test to produce a clear eval- 
uation of the effectiveness of training. Although such an 
extension of this test would certainly provide more meaning- 
ful results, it is beyond the realm of this paper to attempt 
even an approximation of this standard value. 

MicmrecnacdiilenankeCorrelation Coefficient 1s another 
measure of association type test that is appropriate here 


Dut will not be demonstrated in this paper. 
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D. SUMMARY 

This concludes a presentation of possible nonparametric 
meer rsoptcal tests which could be used in evaluating the 
effectiveness of military training. The results of the two 
tests conducted in this chapter appear to be conflicting. 
The Kolmogorov-Smirnov Test suggested that training emphasis 
was not being devoted to the proper instructional areas 
while the Spearman Rank Correlation Coefficient indicated a 
degree of association between training emphasis and job 
importance levels. While it is difficult to explain the 
cause of these differences, it is certainly safe to say 
that the assumptions of the Spearman Rank Correlation 
Coefficient are most closely adhered to while the violation 
of the continuity requirement in the Kolmogorov-Smirnov 
Meee may tend to discount tne statistical validity of its 


results. 
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eect x METHOD POR TRE EVALUATION OF TRAINING 
CUR ecumUM EFRECTLVEUESS 
ieee LNTRODUCTION 

The matrix method of training curriculum analysis pre- 
Semued here has been adopted from a study conducted by 
Peemmur 2. siegel, Douglas G. Schultz, and Phillip Federman 
in 1961 for the Office of Naval Research [7]. 

This correlation oriented procedure enables the analyst 
Mmemmeeoermine a training index, an undertraining index, and 
emo verteraining index for each curriculum under evaluation. 
The numerical value of each index will be a number greater 
meam@eor equal to zero and less than or equal to one. The 
Means index is representative of the overall effective- 
Bes: of the program while the overtraining and undertraining 
indexes represent the direction of problem areas if the 
Pi@eeieculum is classified as ineffective. A training index 
of one represents the highest possible positive correlation 
between adequate job performance and the associated task 
proficiency levels established by the training program. 
Basically, an effective curriculum content should produce 
a training index close to one while the undertraining 
and overtraining indexes should be near zero. 

Although the matrix method requires some value judgement 
as to the placement of tasks in the matrix cells, its basic 
advantage is the relative ease with which calculations can 


be accomplished. 


36 





B. METHOD 

The procedure is based on the categorization of tasks 
into four groups labeled "very high," "high," "moderate," 
and "low." Separate classifications are conducted for the 
training level associated with each task as well as the 
relative importance of each task to performance on the job. 
Subsequent to these classifications each task is then sorted 
into one of the sixteen cells in the matrix shown in Figure 
3. Cell A, for example, contains tasks which are very 
highly important and for which trainees have been trained 
eO a very high proficiency level. Cell H contains tasks 
Which are greatly important but received little training 
emphasis. By assigning weights to the cells of the matrix, 
Mmeemebmicee Indexes can be calculated as follows: 

1. Training Index (TI): If all tasks were instructed 
to a level equivalent to that required for job performance, 
one would anticipate all of the tasks to locate in cells 
Mier ana ©. itnese cells are assigned a weight of three. 
Pemls ),G,L,E,J, and 0 are assigned the weight of two, cells 
C,H,I, and N are assigned the weight of one, while cells D 
and M are assigned the weight of zero. Figure 4 illustrates 
the assignment of weights to the cells of the matrix. 

The urriculum Training Index is determined by the sum 
of all the weights associated with the tasks of a particular 
curriculum divided by three times the number of tasks that 
are being evaluated. The mathematical expression 18S as 


re iilows : 


Sf 





Frequency of Performance 


Very High eh) Moderate Low 





Very 





Low Moderate High 


tMeeure 4.0 Matrix Scheme for Task Classification by 
Frequency of Performance and Training Level 


ice UetiC yon seer Oormance 
Very High High HeGderare Low 


Very 





Low Moderate High 





Figure 4, Weights of Matrix Cells for 
Training Index Determination 


38 





me 
1=1 
TI = 
3N 
where: 
Wo = the weight assigned to the cell in which task 
i has been classified. 
N = the number of tasks being classified. 
2. Overtraining Index (OI): The realization of a 


relatively low training index indicates the appropriateness 
of measuring the direction of ineffectiveness in the train- 
ing program. Basically, this means a determination of the 

extent to which individuals are being overtrained or under- 

ine med , 

A measure of the tendency toward overtraining can be 
determined by the assignment of weights to the matrix cells 
femomownm in Figure 5. Only the cells that indicate over- 
training have been assigned a weight. Computationally, the 
overtraining is as follows: 

N 
r Wy 


i=l 
3N 


OI = 


idecmibyewan eltecuive training curriculum will produce an 
awetviraining index of zero. 

Be UNdertraining Index (Ul): This index represents a 
measure of the tendency toward undertraining and is deter- 
mined in accordance with the principles shown above. Matrix 


cells are weighted as illustrated in Figure 6. Only the 
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GY) etal 


Low Moderate Hi 


Figure 5. Weights of Matrix Celis for 
Ovemercuaine Index Detvermination 


ge Uc ey ot Performance 
Very High Hign Moderate Low 


Very 


Low Moderate High High 





Figure 6. Weights of Matrix Cells for 
Undertraining Index Determination 
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cells indicating undertraining have been assigned a weight. 


Sompucavionally, the Undertraining Index is as follows: 


The three indexes for each training course, taken toge- 
ther, provide a useful summary of training effectiveness in 
each instructional area. While the indexes themselves 
provide a valuable indication of the effectiveness of train- 
Mieeecne task classification procedure is more specific in 
pointing out where increases and decreases in training 


emphasis are required. 


C. ANALYSIS AND APPLICABILITY 

Pome policCaypilon ©f the matrix method of evaluation, 
the three indexes were calculated for each of the courses 
under study. The results are shown in Table V. The 


training level classification for each task was provided by 


TABLE V 


reowitoeer  ThR MATRIX METHOD OF EVALUATION 








Treen Overtraining Undertraining 
Basic Ie 2073 me 
Administrative .813 094 094 
Senior noes miles yO Sc 
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Pen airectors Of the training program, while the frequency 
of performance level was determined in accordance with the 


criteria displayed in Table VI. 


TABLE VI 


Poor REGUBNCY OF PERFORMANCE CLASSIFICATION CRITERIA 


Percentage of Time 


Glassificarvion Devoted to Job 


ete net Cla i adele Oo 
High 6 - 14.99 
Moderate Bove 15) ASI, 
Low O- 1.99 


HOreume training courses being evaluated, the three 
indexes each view the data in a different manner. When con- 
sidered collectively, they provide a thorough indication of 
the extent to which the training program is preparing its 
emacdwieatves fOr performance on the job. No one index can be 
used to determine absolutely if one course is more effective 
than another, nor is any one index necessarily the one by 
which a particular program should be evaluated. Basically, 
all three indexes must be considered together and in rela- 
Prom tO One another. 

Without complete knowledge of the training objectives 
and the administrative constraints associated with the 
Maco nenOtcanis tt 1s almost impossible to assign a stan- 


fanvoeitenetnemindexes should have met in order for the 
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fieses tO De considered effective. However, from a more 
academic standpoint, the deviation of the training index 
from 1] and the undertraining and overtraining indexes from 
Zac oroe Cather small for the three courses being 
examined. Therefore, a tendency toward an effective rating 
memoussested, although an effective-ineffective judgement 
at this stage of the analysis is somewhat impractical. 

A thorough examination of each task located off the 
Mame colagsonal in the classification matrix would have To 
memconaucted in order to determine if adjustments in 
Geagnane emphasis would be feasible. Nevertheless, the 
training indexes as presented increase the understanding 
of the effects of the training program and provide a 


Meeicatvavive basis for altering training emphasis. 
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Vier RC METHOD 


A possible parametric procedure for determining the 
G@uality of the training program is the student's "t" test. 
moesUsetulness of the test is limited to an effectiveness 
Evaluation on each individual task element. Basically, 
teeallowS a statistical test to be conducted on the extent 
Co which each task element of a training curriculum is job 
meeecG. from the calculation of test statistics for 
each task statement and the subsequent determination of 
Pots Cal significance, one could make an accurate 
analysis of the emphasis of the training content. Because 
feeeune large number of calculations involved, it is 
beyond the scope of this paper to employ the parametric 
procedure mentioned above. This chapter will be devoted 
tO an explanation of the procedure and the assumptions that 
are required for using the "t" test in a meaningful way. 

The problem becomes one of testing the hypothesis that 
the mean of a population equals a specified value against 
the alternative that the population mean is not equal to 
the specified value. The hypothesis is expressed as 


follows: 
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where: 


Ws mean importance level of task i 


u = established proficiency level associated with 
o1 . ane ; 
Vi Teire ne Frainineg curriculum. 


The test is based on the reasonableness of the assumption 
that the population of importance levels are normally 
Gistributed [2]. If this assumption is feasible within a 
reasonable degree of approximation, then the test statistic 


is as follows: 


obs ae 
ime re 
X; = the mean of the sampled population 
S = the variance of the sampled population 


mn = the number of job incumbents from which 
data was collected. 


Miomer| vical yalue is extracted from any standard "t" table 
and is based on n-1 degrees of freedom as well as the 
desired level of significance. If t observed is greater 
Pictmee critical, the null hypothesis is rejected and it may 
Memos Toonmunav training is not job oriented for the task 
element tested. 

This same procedure is then repeated for each element 
of the program so that either an acceptance or rejection 
of the null hypothesis is accomplished for every task 
Searement:. 

The t test used in this manner has the advantage of 


evaluating the significance of every element of the program 
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and provides accurate statistical information for altering 
the emphasis of course contents. Since a large number of 
task elements imply extensive calculations, computer 


techndiques are suggested for computational efficiency. 
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VI. AN OPERATIONAL APPROACH TOWARD TRAINING EVALUATIONS 


Previous chapters of this paper discussed nonparametric 
and parametric procedures for evaluating the effectiveness of 
military training. Alternative approaches were illustrated 
providing statistical techniques for analyzing the extent to 
wach Craining is job oriented. These procedures provide 
insight into the emphasis of course contents based upon 
Meevern SvtCavistical methods. However, it still remains 
impractical to assign an effective-ineffective rating to any 
means Curriculum, and therefore, evaluations based on the 
operational significance of the data are also required. 

As a possible approach toward an appraisal of the train- 
ing program in a more operational manner, it appears 
appropriate to consider the classification procedure demon- 
strated in the chapter on the matrix method of evaluation. 
The statistical procedures demonstrated in Chapter III may 
provide equally acceptable starting points; however, only 
the matrix method will be pursued here. Tasks located 
ao eonewmain Giazonal of the classification matrix are 
representative of candidates for a more intensive examina- 
PEonmof preper training emphasis. Those tasks assigned to 
Comers aN eana M Of tne matrix should, therefore, be 
of particular concern to the directors of the training 
program, 

Subsequent to determining candidate tasks for a more 


detailed analysis on an individual basis, it seems 
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appropriate to examine the job data for trends or patterns 
based on the nature of the selected tasks. Comparisons of 
these trends or patterns associated with the importance of 
the tasks to performance on the job against those related 
to training emphasis provide information as to the timeli- 
Mess Of Che training program. More specifically, it is not 
only important to determine what training should be given, 
but when it should be given in order to be timely. Dis- 
crepancies at this stage of the analysis indicate an 
untimely training program where individuals are either being 
trained prematurely or subsequent to the time that they are 
required to perform the task. 

As an example of the timeliness concept, consider a 
complex task not normally required of lower ranking indi- 
viduals but continuously becoming more important as the 
individual increases in rank. A timely training program 
wommldad not provide detailed instruction on this task for 
the entry level ranks but would emphasize the task within 
Ziemi@ore saavanced COUrses. 

MEperations: 1m training empnasis indicated by an 
untimely training program would include the elimination of 
instructional areas from one course while adding them to 
another. In keeping with the objective of military train- 
ing, these additions and deletions should be accomplished 
in accordance with the importance of the tasks to be 
performed on the job. Other tasks receiving timely training 


may still require modifications in emphasis in order to 
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bring the program more on target. Adjustments in course 
membenlvs, when the timeliness of training is not an issue, 
require an examination of the courses on an individual 
Dasis since a task receiving proper emphasis in one course 
may be overemphasized or underemphasized in another. Hore 
eect tically, the evaluation procedure suggested here is 
associated with determining when training should be accomp- 
mesmed LO be timely as well as determining the proper 
paemasis that Should be devoted to specific instructional 
areas within individual courses. 

As an example of the concepts discussed above, consider 
task Numbers Three and Sixteen. The tasks are listed on the 
job inventory questionnaire displayed in Appendix C. Task 
fameer Three is "the assignment of work to personnel on a 
feeeaebesis.' The data also indicates that the frequency of 
performance of this task is continuously increasing as 
mcieiauals advance in rank. Because task Number sixteen 
igs more menial in nature, its associated importance level 
decreases in accordance with the rank SeUrucuure. 

As previously mentioned, the job inventory data was 
ee pamaved into three distinct categories representative of 
the Marine's most recent course completion. Table VII 
illustrates the range of ranks associated with each category 
as well as the average percentage of time devoted to task 
Number Three on the job. Table VIII shows the average per- 
centage of time devoted to task Number Sixteen within the 


same three categories of data. 
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AAS aes vale 


SUMMARY OF JOB INVENTORY DATA COLLECTED 
FOR TASK NUMBER THREE 











Data Category Rank % Job Time 
Basic E2-E3 Aol 
Intermediate E4Y-E6 el 7 
Senior E7-E8 1.23 


An observation of the curriculum contents reveals the 
Seme cOncinuous tendencies as illustrated in Table VII and 
Table VIII. These results are indicative of a timely 
training program in which individuals are instructed at 


the proper rank for adequate job performance. 


TABLE VIII 


SUMMARY OF JOB INVENTORY DATA COLLECTED 
FOR TASK NUMBER SIXTEEN 





Data Category Rank % Job Time 
Basic ‘E2-E3 8.54 
Intermediate E4-E6 Gyo 
Senior Eee B43 


In contrast, task Number Eleven as listed on the ques- 
tionnaire displayed in Appendix C, involves the maintenance 


of financial records. The data shows that this task is 
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performed more regularly by lower ranking individuals while 
training emphasis is more concentrated toward the senior 
enlisted man. Task Number Eleven is therefore representa- 
tive of an untimely aspect of the training program and 
should receive a more intensive examination. 

Once tasks have been evaluated for timeliness as demon- 
strated above, it still remains to examine the emphasis 
devoted to them within each individual training course. 
Mptsmperolon Of the analysis involves the modification of 
Beurse Contents in accordance with the importance of tasks 
associated to performance on the job and could be accomp- 
lished by adjusting training emphasis enough to relocate 
these tasks on the main diagonal of the classification 
Mearrix., 

However, it still remains to consider the administra- 
tive type constraints associated with formal military 
Matalin prior to the adjustment or reorganization of course 
contents. Since these constraints are obviously varied and 
MmiWemcGonrspeciti1c training programs, they will not be 
included here but are considered an appropriate area for 
future research. Although a specific judgment of the 
effectiveness of the program remains impractical, a more 
operational approach toward the employment of task analysis 


in the evaluation of military training has been introduced. 
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VII. A GENERAL METHODOLOGY 


This chapter introduces a general systematic methodology 
PereuUcilizing task analysis in the evaluation of military 
training. The major features of the proposed methodology 
mevememerrsed irom modern training research and include 
statistical techniques, operational evaluations, and compari- 
sons of data collection methods. The procedure consists of 
ten steps to training curricula evaluation: 

1. Conduct a systems analysis of the program To be 
evaluated, thereby determining the rank and MOS requirements 
Suecenie individuals to be employed as subjects. 

2. Determine the size and location of a representative 
wales Oor job incumbents from which job specification data 
can be obtained and consider sample stratification techniques 
Pempenecasiple to include the entire population. 

3. Develop the task analysis inventory to be adminis- 
tered to job incumbents. The foundation for this development 
comes from the systems analysis of the training program as 
well as established job requirements. 

4, Select the data collection method to be used and 
conduct the task analysis. 

pee eolecte une appropriate statistical procedure. 

6. Consolidate the job data so that a comparison with 
Piemonte promncr1ency levels can be conducted in accor- 


Gemcemw ren une Stavistical technique selected. 
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{. Conduct the statistical test and determine if sta- 
bastical significance exists. 

8. Examine available data from an operational stand- 
point to include a consideration for the timeliness of 
meaning . 

9. Adjust the training program where feasible. 

mtre Monitor graduates on a pericdic basis in order to 
Maintain task analysis data in a current status. 

The methodology suggested here is certainly general in 
nature and presents no revolutionary ideas in the search for 
an efficient way to evaluate training. It does, however, 
provide some procedural guidance for using task analysis in 


the evaluation and revision of military training curricula. 
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a or DAT LON Or DATA COLLECTION METHODS 


As mentioned in Chapter I, a secondary objective of this 
Meaeetets tO attempt, on a relatively small scale, statisti- 
cal comparisons of some of the data collection methods 
foeoeelaved with the Job inventory questionnaire. This par- 
ticular aspect of task analysis will be used to determine if 
meee OObainead Irom any one collection method is signifi- 
Gaely Giilterent from that obtained by one of the other 
procedures. Chapter I also lists the methods of data col- 
lection most feasible for Marine Corps task analysis Beers 
Meee Of these four methods were subjected to a test to 
determine the degree of association existing between the 
Emreee distinct combinations of data gathering procedures. 
PMiemeask analysis data collection methods selected were the 
Meewereasiole for this type of comparison and are as follows: 

woe verunea Number 1 — Personal interview and actual 
Seservabion Of an individual's on the job performance. The 
investigator completed the questionnaire based upon verbal 
responses of the job incumbents. 

2. Method Number 2 - Distribution of the task analysis 
questionnaire in a manner not permitting observation or 
interview of the job incumbent. The workers completed the 
questionnaire without assistance from a trained observer. 

Ce erode Numberss— Distribution of the questionnaire 


temper visers reporting On the job specifications required 
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of their subordinates. Supervisors completed the question- 


Meme WI Chout Girect assistance from a trained observer. 


A. DATA COLLECTION 

The data for this segment of the paper was gathered over 
a period of about six weeks. Fifteen Marines from the 
Marine Corps Supply Center, Albany, Georgia, were employed 
as subjects. The Marines held a rank of either E-3 or E-4. 

Initially, staff non-commissioned officers and officers 
completed the inventory questionnaire on the job character- 
istics that they felt were required of their men. The 
questionnaire was at the same time distributed to the fifteen 
gob incumbents in accordance with data collection method 
Number Two. The subjects received a brief explanation of 
the instructions and were then given a period of about 
twenty-four hours to complete the questionnaire. No explan- 
ation of the individual task statements was permitted 
although precautions were not taken against subjects dis- 
cussing some of the statements or procedures prior to 
marking the questionnaire. 

Podciebvencartver a period Of about six weeks, the subjects 
were interviewed on the job. For this portion of the 
experiment the subject considered each task statement on 
the questionnaire with the investigator and actually was 
required to explain why he felt that particular tasks were 
required in the performance of his job. The investigator 
then marked the questionnaire in accordance with the res- 


ponse of the job incumbent. 
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The same questionnaire was employed throughout the test 
and is shown in Appendix C. In order to limit the selection 
Of responses in regard to the extent to which specific 
asks are required for performance on the job, a rating 
scale containing only four points was used. An explanation 
of the four point selection scale is also presented in 
Appendix C. 

Peditionally, a questionnaire useful in evaluating the 
eare@tevo Which human factor characteristics or worker 
characteristics are required for successful performance on 
the job was administered to these same subjects at the same 
Piiewas tne perlormance Cask questionnaire. This particular 
Questionnaire was explained in Chapter I. and was developed 
by Professor Gary K. Poock, a member of the Operations 
Analysis faculty at the Naval Postgraduate School. This 


questionnaire is displayed in Appendix B. 


B. DATA ANALYSIS 

Observation of the completed questionnaires revealed the 
development of an apparently significant trend in regard to 
the three data collection methods. The distribution method, 
in which individual workers were required fo complete the 
questionnaire on their own, without detailed explanations of 
Wiimiiseruiet onc wey lelded higher total point counts in all 
cases when compared with the investigator conducted inter- 
eo at Omomeoiermeotal DOlnt Count in ten of the fifteen 


cases when compared with the supervisors reports. 
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Additionally, the results of the questionnaire when used by 
supervisors to report on their subordinates (data collection 
method Number Three) demonstrated a higher total score for 
fourteen of the fifteen subjects when compared with the job 
interview method administered by the investigator. Total 
point count was obtained by adding the responses of each 
Subject on both of the questionnaires that were used in each 


phase of the test. 


GeeeteolinG FOR STATISTICAL SIGNIFICANCE 

The nonparametric Sign Test was used to test for statis- 
tical significance between the pairs of data gathering 
procedures. Because of the large sample size of task 
statements the normal approximation to the binomial distri- 
Meron was required for computation of the test statistic. 
For a detailed discussion of the rationale and method of 
this test see [6]. Three separate test statistics were 
eemewlared so that the following statistical comoarisons 
could be made: 

1. Method 1 versus Method 2 

ew Method 1 versus Method 3 

8. Method 2 versus Method 3 

The following null hypothesis and alternative hypothesis 
were used throughout the testing: 


Melneeeinere 1s 00 Ssig@niticant difference between the 
data gathering methods. 


[eee te lsmanorenaiieant difference between the 
MetioGs . 


The results of the tests are indicated in Table IX. 


Sif 





PASE Six 


SUMP Ole thE SGN ThsT RESULTS ON THE 
PolrnowOon wet AP COLLECTION METHODS 





Test Critical Value P Value 
IL Wey Bm COANE, 
ievce 5 Bec .0026 
Pes , 1.53 .1260 


Statistically, the results of this test are determined 
by comparing the P value with the desired significance level 
(a). The null hypothesis can be rejected if P is less than 
or equal to the selected significance level. AS an example, 
meen —- 2.005 15 chosen, the results indicate a significant 
difference between data collection method Number One and the 
other two procedures. No statistical significance is 
pemonst rated between methods Two and Three. Statistically, 
the results indicate that job data obtained by distributing 
OheobroOnnaires to job incumbents is not different from that 
collected by supervisors using the questionnaire to report 
Semele w,ob specifications required cf their subordinates. 
Additionally, the results of the Sign Test indicate that 
the data obtained by personal interview and actual observa- 
tion of the job incumbents is different from that obtained 
using the other two methods. The Sign Test, however, does 
not indicate which data collection method produces the most 


Void aniormat lon 
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D. ADDITIONAL INFORMATION 

IMmaccdtibbommbe: Le Statistical results of the Sign Test, 
it is of interest to consider some specific items on the 
worker characteristic questionnaire in an attempt to deter- 
mine which of the tested data collection methods produce the 
most valid information. Since this questionnaire was 
designed for use on all MOS's, some questions are completely 
facewevyant tO the type jobs considered for this evaluation. 
Of particular interest is Question Twenty-nine which querries 
the subject as to his need for an accurate sense of taste, 
and Question Fifty-five which asks the worker if the ability 
to cause subordinates to willingly produce desired results 
is required in the performance of his job. These particular 
worker characteristics are representative of those definitely 
Mee required of any of the individuals involved in the test- 
ing. None of the job incumbents was in any way associated 
Peo asvuing food in the performance of his job. Addition- 
ally, none of the subjects had the responsibility of sub- 
ordinates nor were they required to possess the capacity to 
iacmoredi rect Others in the performance of their jobs. 

In this regard, it appears significant that eight of the 
fifteen subjects answered question twenty-nine with a res- 
ponse greater than zero when the questionnaire was adminis- 
PemeCmuramcaistribuLlon and without detailed verbal instruc- 
blons. Additionally, fourteen of the fifteen subjects 


responded with a score greater than or equal to two on 


a0 





question fifty-five when this same data collection method 
Tame moloyea.. Olner Similar inconsistencies were evident, 
revealing that data obtained via distribution of the task 
analysis inventory could not be considered valid. Similar 
although less frequent inconsistencies appeared when the 
questionnaire was distributed to supervisors who employed 
Memeo repoOrte On the owrker characteristics required of their 
smoordinates. 

PEpOossiple explanation of these results might be 
Geernmouced tO the hesitancy of any one to admit that very 
few specific characteristics are required in the performance 
@emois JON. Quite naturally, humans associate with their 
jobs a complex set of procedures requiring a variety of 
human characteristics as well as a vast number of learned 
perl 1s. 

Although the number of subjects employed was small and 
Meuebepresentacive of the entire population, the personal 
interview appears to be the superior data collection method 
of those tested. Regardless of the explanations or conclu- 
plowmomaneolved, this portion of the paper represents an 
area of task analysis research that could and should be 


expanded. 
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IX. SUMMARY AND CONCLUSIONS 


This paper has investigated some of the problems of 
present day training evaluation techniques, and has been 
wore) cO describing methods for analyzing training 
Sitectiveness using valid task analysis inventory data. 
Hopefully, it may assist in bringing training emphasis more 
ere career. 

Task analysis data was collected by the investigator 
and subjected to parametric and nonparametric procedures 
momeooaclstcaically determining the extent to which training 
Was jJOb oriented. Of particular interest was the matrix 
method of evaluation, which appears to be the most flexible 
procedure based on the nature of available data. 

A general methodology for analyzing training programs 
using task analysis was presented. Included in this method- 
ology was the importance of operational considerations such 
as the timeliness of training. In this regard, an evalua- 
MEOmnappreach Originating with the classification of tasks 
Pameme Matrix Evaluation method was presented. 

Although three specific supply oriented training vro- 
grams were subjected to the analysis techniques presented, 
ibeeveaswOovrously ampractical to assign an effective or 
ineffective rating to any one. A study of the data did, 
however, suggest the necessity for some changes in training 


emphasis provided the administrative and cost constraints 
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could be satisfied. Of particular interest was that the 
data revealed overtraining as significant a problem as 
undertraining. Although cost estimates were not obtained, 
this particular aspect of training analysis could and 
should be expanded as a possible approach toward signifi- 
eae training cost reductions. 

Additionally three distinct questionnaire oriented data 
collection procedures were subjected to a small scale 
experiment to determine if any one method provided the 
Heweevallid anformation. Although this experiment was based 
on a sample size of only fifteen relatively low ranking 
enlisted Marines, it was concluded that data can best be 
obtained by trained observers completing the proposed 
questionnaire based on verbal responses from the job 
incumbents. The possibility for future research in this 
meee Was also indicated. 

Pomiabiy, thas paper has attempted to indicate the 
importance of improved procedures for determining the 
adequacy of training content. Curriculum development would 
certainly profit from the use of task analysis data as 
feedback to determine whether training programs and objec- 


tives satisfactorily meet job specifications. 
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APPENDIX A 
CURRICULUM DESCRIPTIONS AND ASSOCIATED 
RANK REQUIREMENTS 

BASIC SUPPLY COURSE: This course is designed to train 
Privates (E-1) through Lance Corporals (E-3), who have 
mmscved Or no supply experience, with only the basic funda- 
Memeals Of the technical skills required of supply clerks. 
Basic skills taught include the preparation of routine 
eieeespondence and reports, use of supply publications and 
aeee@e 1150S, computation of allowances, and a familiariza-— 


tion of basic property control and small purchase procedures. 


See ot SA ODMEINTSTRATIVE COURSE: This course is designed to 
train Corporals (E-4) through Staff Sergeants (E-6) with 
some of the formal principles, procedures and techniques 
related to the supply field. The course provides an inter- 
mediate level of instruction. Basic tasks taught include 
the following: 

1. Additional emphasis on the tasks included in the 
Basic Course. 

Popo ly sOperautonal, elerical;, and administrative 
Mitcocminetaetit tO requisitioning, procuring, and accounting 
for supplies and equipment. 

3. The duties of a stock reviewer, maintaining card 
files, as well as screening and offsetting transaction cards 


WeuheapprOpra1ate item record cards. 
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4, Mathematical operations on the electric calculator. 

>. ithe general operating principles, uses, and capabil- 
ites Of Supporting EAM/EDP equipment. 

6. Interpretation of computer produced reports. 

{. Preparation of correspondence, messages, and reports 


ieaehormat for future mechanization. 


SENIOR COURSE: This course is designed to provide senior 
staff non-commisioned officers with a more complete under- 
standing of the entire range of operational principles and 
methods which contribute to effective management within the 
following specific task elements 

1. Supervising the duties of enlisted personnel. 

PemeCecordanaving the daily supply activities of a unit 
Grmoreanization,. 

3. Supervising the duties of personnel involved in 
Miveemasing and contracting activities of a unit or organi- 
ZaGOn , 

4. Possessing a general knowledge Gime pRece CURES, 
directives and regulations applicable to the functional areas 
of the supply system. 

5. Assisting in the determination of supply and inven- 
tory management programs. 

6. Possessing the required knowledge for the adminis- 
Meterommcameanoperatton Of a@ Supply activity or organization. 

7. Having an understanding of basic computerized opvera- 


tions and their application to supply management techniques. 
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APPENDIX B 


HUMAN FACTORS IN TASK ANALYSIS 


peumber Of the human characteristics listed below may 
be needed in the satisfactory performance of the tasks 
included in your job. The accompanying form is designed to 
Bees your estimation of the necessity of the character- 
mowees, which are referred to as Worker Characteristics. 

Mechnas Seelion Gevoced to Worker Characteristics, 
indicate, by the placement of an "X" in the appropriate 
[ewe une Cdepree of Gach characteristic you believe 
Meecessary to do your job. The definitions of the several 
degrees are defined as follows: 

Soyo lnis characteristic is not required. 

(1) This characteristic is occasionally required. 


SE 


(2) This characteristic is frequently required, but 
not on a regular repetitive basis. 
(3) This characteristic is regularly required, the 


lack of which renders your job virtually impossible 


Om Oe Orm, 
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WORKER CHARACTERISTICS 


Level 

Reauired 

Oia 23 
ime PHYSICAL RO aa ony 


A. STRENGTH: In the four categories below, 
pone ween sSculame Strength. not muscu- 
lar endurance. Considerations: The size 
weight and bulkiness of objects handles; 
The pressure necessary to operate con- 
GOs mrad s OOS 


PTellUcer mend, wrist and forearm 
SUceue oie — hay necessary to 
SO UGSZE , INGING . JeUbil sy wwiiaiis eS miblie ole 
shape or grip objects. 

MP vleoe a uiesmicieria—— Tnar necessary 
oman oswi Mee push epull. carry or 
throw objects. 

3. Back and shoulder strength -- That 
necessary to lift objects from the 
floor, move objects with the back 
and shoulders or swing heavy tools 
COrstrike Obsects. 

4. Leg, foot and ankle strength -- That 
necessary to lift objects using knee 
action, operate pedals with pressure, 
Seip worwomeCemvwlEme Lae wKnees or climb, 
kneel, walk or stand with loads. 

Peel nela: fn the folllowing six categories, 
consider both muscular endurance, the ability 
To sustain strength over a period of time, 
and eirevulo—-respiratory endurance, that 
Caemaepverigtpic commonly referred to as wind, 
VicmobililGowecO suStaln Vigorous activity over 
ao 1 OC Otel Mee ameluding the maintenance 
of strength. Consideration: Pacing by 
machines or superiors; the frequency, dura- 
migimmciCemicapralyy, Gf movement; the extent 


SUemereiouleamacuivaty such as lifting, 
Mimi Neerawlang, leaping, etc. ; 
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Level 
Required 


OFt.2 8 


Vnietie iw tnerauralion Of the activity is 
known or indeterminate length. 


5.» Rapid work for a series of short 
Dew sOdsr 

6. Rapid work for extended periods. 

(om hep le workmeiOr indetinite periods. 

Lee vweavorie@mterra series of short 
PemwnoGis. 

Per cian WevOGiembor extended periods. 

10. Heavy work for indefinite periods. 


DEXTERITY: The three categories immed- 
iately below rate tne skill or adroitness 
in the movement of the subject parts of 
the body. Considerations: the speed, 
complexity, and repetitiveness of the 
movements; the accuracy required; 
whether or not all digits or limbs are 
used to complete the movement. 


lil. Finger dexterity -- Ability to move 
and manipulate objects. 


12. Hand and arm dexterity (including 
| the fingers as part of the whole). 
Ability to move hands and arms 
accurately and quickly. 
jee OOuscand Vertndexverity —- Ability 
to move feet and legs accurately 
ENG Giles. 
COORDINATION: Considerations in the fol- 
lowing five categories include the 


frequency, complexity and repetitiveness 
of movements. 


14, Eye-hand coordination -- That neces- 
cdi em comuico Uncine hand Chrough the 


WSe. el vision. 


Oi 





6), 


ls 


nS 


Foot-eye-hand coordination -- That 
Ne@ecoary ve concerol, Simultaneously, 


Che feet and hands through the use 

Oey rser. 

Coordination of the independent 
movement of both hands -- That 
Mececoctn mUOnConure), simultaneously 
and independently, both hands, with- 
SUm—~eccoccr lily Usine the eyes. Addi- 
tional considerations are the distance 
the Hands move and the differences in 
the movement of the two hands. Each 
hand may be doing something different. 
ReGwec omeOOrdiharaon =— That meces— 
come VOncOMermolsenew1ecet through the 
SE oid ak tsaleyaye | 

Foot-nand coordination -- That neces- 
San MUOmecoOmu re mmaniniG mice l. and tands 


S MMe Oust mOmnUSime the Eyes . 


Pez CONST DE RATIONS: “Any DbDoundaries on 
Velesoiyaweat s14e,, Upper or lower, 


Mececolvavedeby phe task pertormed, not 
by recoikat ions. 


hones Chive iiss dita “Dl arale) 


ZO), 


Weight. (fill in blank) 


WORKING CONDITIONS: 


ere. 


Unpiecacdiemeconcertemds —— 21O work 
NndemRmCceomaciriens olrecting physical 


comfort. These include bad odors, 
excessive noise, vibration, dust, 
iG UNeSs ser Nols lure, high 
Miter coy ReMes 12a  EeMperature , 
constant flux or a wide range in 
temperature, exposure to acids, 


unpleasant sights. 
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Level 


Required 
Ora 55 


ee. Hazardous conditions -- To work under 


conditions affecting physical safety. 


ite oeNSORY CHARACTERISTICS 


tee VLSUAL: 

23. General visual acuity -- That neces- 
sary to perceive or recognize 
SileCeseOcete Points at a distance 
or make accurate discriminations 
Hs tives Ol) 

24. Color discrimination -- That neces- 
Sacyero CIscImeuish similarities and 
differences in colors and shades 
Chereof, create harmonious combina- 
MlowmecmOmuCOlOmmorsilnx Or Match colors. 

B., VISUAL - MENTAL: The five categories be- 
low involve judgments requiring the use 
of vision. Considerations: the complexity 
of the objects perceived; the frequency 
and rapidity of the required observations; 


their variability; any mechanical aids 
used. 


25. Size estimation -- That necessary to 
make accurate judgments of dimen- 
sions (height, weight, depth, breadtr 
or thickness) or over-all size, area. 

26. Quantity estimation -- That necessary 
to make accurate judgments of the 
MUM ee Or Copecwny Of Gb)ects. 

Zee Oc ecomeoe MarlOnu=—alnar necessary to 
make accurate judgments of rate of 
Caante won POS Melon, involving the 


Sspimeewienwonwboun time and distance. 
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Oo, 


27 


Level 
Reauired 


Ora 2° 3 
Quality estimation -- That necessary © 
to judge the quality or workmanship 
Cie Mave wee hits eCharacteristic 
may very well involve the use of the 
Saemociscsnesuleimas, tCouUen., hearing, 
smell or taste. 
LOwMimocrceerlowm—— ihar, Necessary tO 
distinguish correct shape or outline 


or generally perceive shape. 


AUDITORY 


BGr 


General keenness of hearing -- That 
Necessary TO. recognize particular 
sounds and distinguish similarities 
and differences in pitch, intensity 
enue ia la tino t ss@uUnicds: 


OLFACTORY 


Slee 


sense of smell -- That necessary to 
recognize Particular odors or 
discriminate differences and 
SIMiibecwe Teste Mowaualit LY Or 


invensity of odors. 


TACTILE 


Bie. 


Boe 


ROucimcdLscicimunalbrome=—  Ihnat neces— 
sary to accurately judge smoothness, 
roughness and other surface qualities, 
Ws Hig SCOue ll. 

Muscularediscriminarton ——- That 
necessary to make judgments based 
OMemuUscilLartsemsttaviTty,. SUCH as 
estimating weight or resistance by 
Pari nao woe pI Tne > Sauging 
DOS eOM@mOr selina ne body Members 


without the use of vision. 


{0 





Level 


Reo uUtice Gd 
Ora 32.3 
ee leo. 
34. Sense of taste -- That necessary to 


cme awelverdistaneushi similarities 
and differences in the intensity or 
quality of tastes, or recognize 


Particular tastes. 


IfI. MENTAL CHARACTERISTICS 


A. MEMORY: The five characteristics below 
rate the capacity of the mind to store 
images for future references. Con- 
siderations: the number and complexity 
Smerimatesca netic rate Of accuisition;: 
the length of time the images must be 
remembered. 


Doe eloGy mor Concrete. details, 

36. Memory for ideas, theories, plans, 
MuOcerses yoo MUCTes. 

Pl lewom Vet ommoral Girecvions. 

38. Memory for written directions. 

Soo Coby! Or emaves ange persons. 


See ante D CHARACTERGSErGs: 

40. Arithmetic computation -- To perform 
ealcullations and higher mathematics. 

41, Planning Ability -- To recognize and 
comprehend the steps necessary to 
achieve specific ends, decide upon, 
ScUumUpanadmeoorduimave Such plans, 
and organize ideas and things. 

42, Mechanical Ability -- To understand 
and use the principles of mechanical 
structure and operation and solve 
problems involving tools and machines. 

43, Oral Expression -- To express oneself 
clearly and effectively, in use of 


speech. 


ce 





Hy, 


Level 
Required 


OSes 


Neeevem xpress 1em —-— No express 
oneself clearly and effectively, 


presenting information and ideas 


Ate iresate at Kee 


no oCHOLOGLCAL CONSTDERATIONS 


Jor 


46, 


NT. 


48, 


49. 


50. 


Oil 


Attention to many items -- To keep 
ijimimnicaaiiany parts of one job, 
although required to repeatedly and 
COhismanp ly Shatt one's attention 
PeOMmCaciw tO Part. 

Adaptability -- To adjust readily to 
mew situations; Prlexibility. 
Decision-making ability -- To be able 
to consider evidence, and, with a 
minimum of delay, reach a reasoned 
Cone Tusiene 

Initiative -- To recognize the need 
for a change in procedures and 
aeulOrcwmanamre accomplish Such 
changes without specific instructions. 
Tact == Te Use diplomacy in dealing 
with people to achieve certain ends 
(consider only situations where this 
Cite ep ctl hnemoe tantly rercu lar 
requirement of the job). 


Personal anvearance -- To maintain 





a level of neatness, grooming and 
attire above and beyond that con- 
sidered necessary for a good Marine. 
Concentration amid distractions -- 
To continue performing a task or 
tasks amid moise, interruptions or 


Cuoco rmaecrine ini ivences. 


{2 





Syst 


oe 


54, 


Dee 


Or: 


Bie 


Dor 


Dos 


PFMOpronaleovabllitve—-— ©£o maintain 





self-control and calmness at all 
times. 

Dealing with the public -- To meet 
and deal with the public, not neces- 
sarily involving the exercise of 
reacCtwOMicaMartiGtathiaie friend ly 

eubenE EOS . 

Teamwork -- To necessarily subor- 
dinate one's individual performance 
LOmeio mC OOCmOL bnew becam Or UNIT. 
Leadershinv -- To cause subordinates 
to willingly produce desired results 
through a combination of superior 
knowledge, thoughtfulness, courage 
and exemplary personal performance. 
Depend iitth ys lOmproduce desired 
results at or prior to the time they 
are needed, 

Ei iocc mCeUGa ow ——s LO perform the 
Meco cee aeGge = aigthlesam er POSsiple 
harmful physical consequences. 

Moral courage -- To do what is right 
regardless of the consequences. 
Please list below any other character 
iSotucsa LT eoouUmreecmin your 106 which 
have not been covered above. Then 
Cio carne micve meet hemeiaraclLeri slic 


required in your job. 


is 


Level 


he alee 


OM 2 ess 





ee ee Ole ae 


PERFORMANCE TASK QUESTIONNAIRE 


MicisccumonteDelOow cONsists Of a list of actual tasks 
Pest Dly included in your military occupational specialty. 
Place an "X" in the appropriate column to indicate the 
frequency with which you perform each task. The frequency 
femeaetined as follows: 

Coy nado nou perform this task. 

Ghueelnocriorm this task occasionally. 

(2) I perform this task frequently. It is a regular 

SeaeunOnem: | OD. 
(3) I perform this task virtually to the exclusion of 


others included in my MOS. 


TASK ELEMENTS; Cai 2 =e 


A. PLANNING AND ORGANIZATION 
1. Organize and administer the supply activi- 
mics hol Aner ium  DOsSt Or Station. 
e. Filan and coordinate the daily supply 
Boeivicies, Ol HMEwmtt, post or station. 
3. Assign work to personnel on a daily basis. 


4, Direct the training of supply personnel. 


B. ADMINISTRATION 
WivcC lize Gace COmuentusson Daste directives 
MGmouUpt Lea tonssperualmima=tO Supply. 
6. Utilize publications pertaining to sub- 


Sistence accounting. 
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{. Use the Marine Corps stack lists to ver- 
i eovoc mumbers, Nomenclature, units 
of issue and other data pertaining to 
Sys OI 

APPLICATION 

8. Prepare or edit requisitions, follow ups 
and cancellations for supplies and equip- 
Mie tone a. 

9. Prepare or edit invoices and vouchers 
Bervaining to property control records. 

10. Monitor stock levels and post required 
entries to appropriate accounts. 

ll. Maintain financial records. 

ime oolsb in bNe preparation of a budget. 

ieooeeneconecile fiscal listings and assist in 
the management of funds. 

14. Conduct physical inventories of supplies 
and equipment. 

15. Make adjustments to property records 
Caused DY Inveseleavrons,-i1maVventories or 
Gener Property accountings transactions. 

16. Operate a typewriter. 

17. Operate a desk calculator. 

18. Establish and operate a routine filing 
wes cell. 

19. Maintain correspondence and directive 
WIS < 

20. Employ the procedures for the open market 
Piece Melson sO Ol oomcimnicmeeiialomen tt . 

21. Maintain an allowance list for supplies 
and equipment. 

Bee pare er oubIMme Correspondence and reports 


Deval es svOossupp ly. 


i 


Oe eS 





25. 


24, 


re 


Zo. 


Rc 


Ze. 


29. 


Boe 


Br. 


32. 


ai tathe ih Oricimmaceouny Droperty 
Becords, 

Compute reorder points and requisitioning 
objectives. 

SovaomMictmmeorrcuperylse sa SLOocKk control 
system. 

sereen and offset transaction cards with 
aoiorOoprlave atemerecordad cards. 

Know the general operating principles, 
uses, and general canvabilities of 
Supporting EAM/EDP equipment. 

Interpret computer and EAM produced 
reports and take the required action. 
Prepare correspondence or reports for 
key punching. 

Employ the principles of field ware- 
Mousime tO inclwde the proper care of 
material in storage. 

Bev tami liar Witaeune Droecedures for the 
shipment of Marine Corps material to 
mmekude tie seryarees oifered and the 
PecpOnsi bi lat resmoet commercial §earriers . 
Know the principles and procedures for 
equipment maintenance utilized by the 


Marine Corps. 
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APPEND) 


CUMUGALI VE wole er PUNCTLION DATA 
IN KOLMOGOROV-SMIRNOV TESTS 


BASLE COURSE 


TT 


Task No. Meas mem CDE ) Nene bE) d. 
1 Oe .0018 .0182 
o 02 .0036 Soe 
3 Oe 2 0iZ 0 Oa @) 
4 eee eC aale OOO 
5 .09 ABIES) ea) aleales) 
6 lo . 1086 Ole 
iL w26 SALINE .0984 
8 EO Pe Oe aleite 
9 Por eS EEL 

10 Male) SocwiG ,2o5u% 
al a0 . 3666 21934 
12 56 oo sales 
ILS . 56 . 3986 .1614 
14 56 ~4591 - 1009 
LS Om 5036 .0664 
16 Too .5890 ONL ALG 
ne 60 .6566 .0566 
18 60 .6993 .0993 
19 al . 7456 Pl 350 
20 (6% .7634 .0934 
ol SO ~7794 .0994 
22 3 TO oni B09 57 
23 noe ~8542 OS Sre 
24 . 96 oom .0809 
ED 96 ory On .0809 
26 96 . 8846 Ope 
a .99 Boas 0627 
28 E08 eCioue .0158 
29 lao 00 0 
S10 oe 00 Q 
Bee re 010) 0 
Se eee) aoe 0 

~2254 





Task No. 


bY 
FOO dONAW SWPP 


eo 
I=) 


ADMINISTRATIVE CURRICULUM 


ravine (CDE > 


.025 
6025 
2025 
.025 
.080 
Ook 
nes 
eS 
some 
. 350 
ow 
~435 
JAYS 
Sips 
465 
~485 
oO 
pls 
Toe 
.560 
2565 
s595 
~640 


78 


Job (CDF) 


OBE@ a! 
Oe 
.0596 
.0719 
Oy 
~1453 
AOE 
F2s20 
m0 o> 
~ 3482 
Poe 
. 3863 
~4030 
Zao 
4657 
So 
5838 
sone 
~6625 
SOW. 
TOO? 
ee 
.7968 
Oe 26 
woe 74 
oD 
mere 
.9306 
ee 
5 ed 
lO 
Ls (Xo 


.O149 
7 10 
0346 
~O469 
Ole la, 
050s 
.0054 
nO 70 
0247 
0018 
0260 
0487 
.O420 
0340 
SOOO 7 
.O469 
sOCete 
ko ES 
2s 
rll 
elaine 
2167.0 
» Les 
hi 10 
oe 
a5 
eC ou 
SOG 
eles 
.0661 
~0300 
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SENIOR COURSE 


eeGQ0Re0°=0°0°;°;z;°;¢°;¢°;»;» ; ; ; ; 


Wasik se No. Meainine. CODE) Job (CDF) d; 
1 0294 20351 .0057 
2 PO 2 .0652 .0240 
3 0586 1364 rO77e 
y 0645 . 1886 eal 
5 1294 .25 38 1244 
6 ee m2620 ioe 
7 ea SS ao2ee .0697 
8 em, aoe 0845 
9 . 3063 oe 0839 

10 ee 4062 .0940 
lei: .3712 mies Giant 
ne 24125 4473 PO ue 
lee Yoy3 Nepal £0531 
14 Pigue Peay) gas Nee 
an 4479 Oe 0846 
16 HTS ~5646 laos 
ey ~4479 5997 1518 
18 4479 .6379 .1900 
19 4479 .6790 22) 
20 abe ayaa 2308 
Zl oe 5 ELE: P2640 
22 5480 8014 eee 
23 5598 ea 2676 
Ou -5598 ~8334 .27 36% 
25 m6 bao nos 5 .2406 
26 6129 8625 2406 
DY. . 8666 nOOe 7 SO 
28 9138 .9578 0440 
29 oa oe ROFuD 0257 
30 9492 9799 mes Ole 
2 128 .9900 OIL 97 2: 
22 OO elo 0.0 

*D = .2736 
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APPENDIX & 


TASK TRAINING AND JOB IMPORTANCE RANKINGS FOR COMPUTATION 
OF SPEARMAN RANK CORRELATION COEFFICIENTS 


BASIC COURSE 


Task No. Treainane Hank Job Rank 
il 12 255.5 
2] 25 ee 
3 25 18 
y 25 30 
5 5 3 
6 iL 23 
i 2 y 
8 3 1 
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aS i NS 
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30 25 30 
21 as 30 
52 25 30 


80 





ADMINISTRATIVE COURSE 
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5 30.5 S75 
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6 23.5 a 
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ie 12.0 2u.5 
ie 23.5 PaO 
14 as ne 
eS ne ores 
16 19.0 3.0 
1) 16.5 7.0 
18 eu. 0 ae 
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20 ieee 70 
21 Dae 15.0 
Pe 14.0 5 
a 9.0 13.0 
2 0 16.5 
25 20n5 29.5 
26 7.5 ene 
27 2.0 rene 
28 a. 0 Gee 
29 0 Woes 
30 4.0 18.0 
Bal 5.0 Paee 
32 10.5 Boo 
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it 26) 5 
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9 5.0 ZenU 
EO 20 26.0 
ie) 6.0 ene’ 
ee oe 0 1S 
Is 2 Ore Gras 
14 2025 Ga 
1D DO Zale O 
ke EAS) 35) Sit 
Is ZnS ee 
18 eon LO 
19 POPS 9.0 
20 JOG 5 eS 
md AW) ie 
22 S70 oe 
25 Z Or. © 1 
24 PES SiO 
25 (ale Zoe 0 
26 Cee ee 
Zi ia 0 
28 ono 6.0 
ao 10.5, Zoe!) 
30 28) 5 yee 
31 De 0 2620 
32 ei 270 
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